CAT: Closed-loop Adversarial Training for Safe End-to-End Driving

Driving safety is a top priority for autonomous vehicles. Orthogonal to prior work handling accident-prone traffic events by algorithm designs at the policy level, we investigate a Closed-loop Adversarial Training (CAT) framework for safe end-to-end driving in this paper through the lens of environment augmentation. CAT aims to continuously improve the safety of driving agents by training the agent on safety-critical scenarios that are dynamically generated over time. A novel resampling technique is developed to turn log-replay real-world driving scenarios into safety-critical ones via probabilistic factorization, where the adversarial traffic generation is modeled as the multiplication of standard motion prediction sub-problems. Consequently, CAT can launch more efficient physical attacks compared to existing safety-critical scenario generation methods and yields a significantly less computational cost in the iterative learning pipeline. We incorporate CAT into the MetaDrive simulator and validate our approach on hundreds of driving scenarios imported from real-world driving datasets. Experimental results demonstrate that CAT can effectively generate adversarial scenarios countering the agent being trained. After training, the agent can achieve superior driving safety in both log-replay and safety-critical traffic scenarios on the held-out test set. Code and data are available at https://metadriverse.github.io/cat.Comment: 7th Conference on Robot Learning (CoRL 2023

Guarded Policy Optimization with Imperfect Online Demonstrations

The Teacher-Student Framework (TSF) is a reinforcement learning setting where a teacher agent guards the training of a student agent by intervening and providing online demonstrations. Assuming optimal, the teacher policy has the perfect timing and capability to intervene in the learning process of the student agent, providing safety guarantee and exploration guidance. Nevertheless, in many real-world settings it is expensive or even impossible to obtain a well-performing teacher policy. In this work, we relax the assumption of a well-performing teacher and develop a new method that can incorporate arbitrary teacher policies with modest or inferior performance. We instantiate an Off-Policy Reinforcement Learning algorithm, termed Teacher-Student Shared Control (TS2C), which incorporates teacher intervention based on trajectory-based value estimation. Theoretical analysis validates that the proposed TS2C algorithm attains efficient exploration and substantial safety guarantee without being affected by the teacher's own performance. Experiments on various continuous control tasks show that our method can exploit teacher policies at different performance levels while maintaining a low training cost. Moreover, the student policy surpasses the imperfect teacher policy in terms of higher accumulated reward in held-out testing environments. Code is available at https://metadriverse.github.io/TS2C.Comment: Accepted at ICLR 2023 (top 25%

MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning

Driving safely requires multiple capabilities from human and intelligent agents, such as the generalizability to unseen environments, the safety awareness of the surrounding traffic, and the decision-making in complex multi-agent settings. Despite the great success of Reinforcement Learning (RL), most of the RL research works investigate each capability separately due to the lack of integrated environments. In this work, we develop a new driving simulation platform called MetaDrive to support the research of generalizable reinforcement learning algorithms for machine autonomy. MetaDrive is highly compositional, which can generate an infinite number of diverse driving scenarios from both the procedural generation and the real data importing. Based on MetaDrive, we construct a variety of RL tasks and baselines in both single-agent and multi-agent settings, including benchmarking generalizability across unseen scenes, safe exploration, and learning multi-agent traffic. The generalization experiments conducted on both procedurally generated scenarios and real-world scenarios show that increasing the diversity and the size of the training set leads to the improvement of the generalizability of the RL agents. We further evaluate various safe reinforcement learning and multi-agent reinforcement learning algorithms in MetaDrive environments and provide the benchmarks. Source code, documentation, and demo video are available at https://metadriverse.github.io/metadrive . More research projects based on MetaDrive simulator are listed at https://metadriverse.github.ioComment: Source code, documentation, and demo video are available at https://metadriverse.github.io/metadrive . More research projects based on MetaDrive simulator are listed at https://metadriverse.github.i

ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modeling

Large-scale driving datasets such as Waymo Open Dataset and nuScenes substantially accelerate autonomous driving research, especially for perception tasks such as 3D detection and trajectory forecasting. Since the driving logs in these datasets contain HD maps and detailed object annotations which accurately reflect the real-world complexity of traffic behaviors, we can harvest a massive number of complex traffic scenarios and recreate their digital twins in simulation. Compared to the hand-crafted scenarios often used in existing simulators, data-driven scenarios collected from the real world can facilitate many research opportunities in machine learning and autonomous driving. In this work, we present ScenarioNet, an open-source platform for large-scale traffic scenario modeling and simulation. ScenarioNet defines a unified scenario description format and collects a large-scale repository of real-world traffic scenarios from the heterogeneous data in various driving datasets including Waymo, nuScenes, Lyft L5, and nuPlan datasets. These scenarios can be further replayed and interacted with in multiple views from Bird-Eye-View layout to realistic 3D rendering in MetaDrive simulator. This provides a benchmark for evaluating the safety of autonomous driving stacks in simulation before their real-world deployment. We further demonstrate the strengths of ScenarioNet on large-scale scenario generation, imitation learning, and reinforcement learning in both single-agent and multi-agent settings. Code, demo videos, and website are available at https://metadriverse.github.io/scenarionet

Post-pandemic assessment of public knowledge, behavior, and skill on influenza prevention among the general population of Beijing, China

SummaryBackgroundThe aim of this study was to assess the knowledge, behavioral, and skill responses toward influenza in the general population of Beijing after pandemic influenza A (H1N1) 2009.MethodsA cross-sectional study was conducted in Beijing, China, in January 2011. A survey was conducted in which information was collected using a standardized questionnaire. A comprehensive evaluation index system of health literacy related to influenza was built to evaluate the level of health literacy regarding influenza prevention and control among residents in Beijing.ResultsThirteen thousand and fifty-three valid questionnaires were received. The average score for the sum of knowledge, behavior, and skill was 14.12±3.22, and the mean scores for knowledge, behavior, and skill were 4.65±1.20, 7.25±1.94, and 2.21±1.31, respectively. The qualified proportions of these three sections were 23.7%, 11.9%, and 43.4%, respectively, and the total proportion with a qualified level was 6.7%. There were significant differences in health literacy level related to influenza among the different gender, age, educational level, occupational status, and location groups (p<0.05). There was a significant association between knowledge and behavior (r=0.084, p<0.001), and knowledge and skill (r=0.102, p<0.001).ConclusionsThe health literacy level remains low among the general population in Beijing and the extent of relativities in knowledge, behavior, and skill about influenza was found to be weak. Therefore, improvements are needed in terms of certain aspects, particularly for the elderly and the population of rural districts. Educational level, as a significant factor in reducing the spread of influenza, should be considered seriously when intervention strategies are implemented

Analysis of an Imported Subgenotype C2 Strain of Human Enterovirus 71 in Beijing, China, 2015

Background: Subgenotype C4 of enterovirus 71 (EV71) is the predominant agent of Hand Foot and Mouth disease (HFMD) circulating in the mainland of China. For the first time, a subgenotype C2 of EV71 named SY30-2 was isolated from a HFMD case in Beijing, China. Since it is uncertain whether antibodies raised against subgenotype C4 of EV71 can protect C2 EV71, it is important to monitor and check the presence of cross-reactive antibodies against new EV71 subgenotypes. To find out the causes for the different NtAb, this study is to investigate the relationships between amino acid residue variations and cross-reactive antibodies against EV71 subgenotypes C2 and C4.Methods: Nucleotide and amino acid sequences from full-length genome sequence of SY30-2 were compared to EV71 reference strains. A microneutralization test was used to detect neutralizing antibody (NTAb) in the sera of subgenotype C4 of EV71 infected cases against SY30-2 and FY17 (a C4 isolate). The 3D structure of the viral capsid protein of SY30-2 was constructed.Results: Genome sequence and similarity plot analyses showed that SY30-2 shared the highest identity with subgenotype C2 of EV71 strains in every fragment of the genome. While the microneutralization test result showed that children infected with subgenotype C4 of EV71 had higher NTAb titers against FY17 than SY30-2 (p &lt; 0.001). The amino acid sequence comparison revealed that four amino acid residues VP1-22, VP1-31, VP1-249 and VP3-93 were highly conserved in subgenotype C4 of EV71 compared with the corresponding amino acid residues on subgenotype C2 of EV71 (p &lt; 0.05). Furthermore, the 3D-structure of viral capsid protein showed that VP1-22, VP1-31 and VP3-93 were located on the surface of virion.Conclusion: This is the first report of an EV71 subgenotype C2 isolated from HFMD in Beijing, China. Only a few antigenic variations on subgenotype C2 of EV71 could have led to a great decrease in NTAb titer. Thus, imported new genotypes and subgenotypes of EV71 should be closely monitored. The efficacy of available vaccines against new viruses should be evaluated as well

Estimates of the True Number of Cases of Pandemic (H1N1) 2009, Beijing, China

During 2009, a total of 10,844 laboratory-confirmed cases of pandemic (H1N1) 2009 were reported in Beijing, People’s Republic of China. However, because most cases were not confirmed through laboratory testing, the true number is unknown. Using a multiplier model, we estimated that ≈1.46–2.30 million pandemic (H1N1) 2009 infections occurred

Pandemic (H1N1) 2009 among Quarantined Close Contacts, Beijing, People’s Republic of China

The attack rate was low, and having contact with an ill household member and younger age were the major risk factors